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(54) Image recognition apparatus and method 

(57) An image recognition apparatus recognizes an 
input locus as a command, a figure and a character, re- 
spectively in gesture recognition mode, figure recogni- 
tion mode and character recognition mode. Regarding 
each recognized result, similarity between the input im- 
age and the recognized shape is obtained. The similar- 
ities are compared with each other, and if the difference 
between the similarities is less than a predetermined val- 
ue, the recognized results are displayed for selection by 
an operator. Then, selected one of the displayed shapes 
is determined as the final recognition result. This enables 
correct locus input even il there is a possibility thai the 
iocus is recognized, in different recognition modes, as 
similar shapes of different functions 
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Description 

BACKGROUND OF THE INVENTION 

Present invention relates to a data input apparatus 
for inputting operation commands, character data and 
other type of data ; using an input pen and a digitizer, and 
a data input method for the apparatus. 

[Related Art] 

Conventional pen-input electronic devices usually 
have gesture-command input function that allows a user 
to pen-input operation commands in character-input 
mode or locus-input mode. 

In gesture-command input, if a predetermined input 
locus is recognized, the command corresponding to the 
input locus operates. To discriminate an input locus from 
simply-inputted iocus data, or to avoid confusion of ges- 
ture-command recognilion with character and figure rec- 
ognition, several limitations are introduced as follows. 

In character-input mode, an input locus hard to dis- 
tinguish from a similar character cannot be used as a 
gesture command, otherwise, a character similar to an 
input locus used as a gesture command cannot be rec- 
ognized. Also, in figure-input mode, shapes often used 
as figures such as O, X, and A cannot be used as gesture 
commands. Further, there is a case where the user 
needs to designate command-input mode or non-com- 
mand-input mode. 

As described above, these limitations are made ba- 
sically to prevent confusion of gesture-command input 
with other-type of input in the same input mode. 

However, in the case where the user designates the 
input mode, the user may need to change input modes 
so often that the input operation is not smooth, meaning, 
the manual mode-change operation will be troublesome. 

As mentioned above, if the pen-input device does 
not have command-input mode, the input loci used as 
gesture commands, otherwise shapes recognizable as 
characters orfigures are limited. For this reason, the user 
cannot use, e.g., shapes which are intuitively recogniz- 
able by the user, as gesture commands. Further, as input 
characters are limited, some characters cannot be easily 
inputted. Furthermore, it is confusing for the user when 
a command is inputted with different gestures in respec- 
tive figure-input mode and character-input mode. 

For example, a gesture command "X" (delete) is ef- 
fective when character-input is not made, however, in 
character-input mode, the input "X" is treated as an al- 
phabet. When a gesture command "y" (undo) exists, a 
Greek letter 'Y cannot be inputted. 

In this manner, the conventional input systems have 
different gestures in different input modes regarding one 
input intention, and this is confusing to users. 



SUMMARY OF THE INVENTION 

The present invention has been made in considera- 
tion of the above situation, and has as a concern to pro- 
5 vide an image recognition apparatus and method which 
distinguishes the purposes of locus-inputs, and recog- 
nizes input loci which are similar but of different mode 
types, without confusion. 

It is a second concern of the present invention to pro- 
10 vide a data input apparatus and method which enables 
pen-input with intuitive and user-recognizable images, 
each reflects user's intention exactly. 

According to the present invention, there is provided 
an image recognition apparatus comprising: recognition 
75 means having a plurality of recognition modes such as 
figure recognilion mode for recognizing an input image 
as a figure code, character recognition mode for recog- 
nizing the input image as a character code, gesture rec- 
ognition mode for recognizing the input image as a com- 
20 mand; recognilion determination means for determining 
one of the recognition mode as available recognition 
mode; similarity judgment means for judging whether the 
image recognizable in one of the plurality of recognition 
modes is also recognizable in another one of the plurality 
25 of recognition modes; and selection image display 
means for, if the image recognizable in a plurality of rec- 
ognition modes, displaying candidate code(s) for a rec- 
ognition^) result, for the user's selection. This construc- 
tion achieves exact recognition of input intended by the 
30 user, when the input image of figure, a character or com- 
mand is recognizable by a plurality of recognition means. 

Note thai the similarity judgment means performs 
similarity judgment based on input judgment reference 
value. 

35 As described above, in a case where a plurality of 
recognition modes are activated simultaneously, even if 
there are similar shapes which are often used in the re- 
spective modes, or even if recognition processing of in- 
put locus cannot be easily discriminated, the image rec- 

40 ognition apparatus and method can perform recognition 
in accordance to the user's intention. 

The present invention is advantageous since the 
user can clearly designate recognition processing to an 
input locus, further, the above-mentioned limitations on 

45 gestures, characters and the like are removed. This 
avoids confusing input operation and eliminates 
time-wasting storing of gestures in different modes used 
as the same command. 

Further, it is not necessary for the recognition sys- 

50 tern designer to consider the limitations on shapes of in- 
put loci. This allows the recognition system designer to 
use any shape. 

Furthermore, in command-input, more intuitive and 
user-recognizable commands can be used. 

55 Other features and advantages of the present inven- 

tion will be apparent from the following description taken 
in conjunction with the accompanying drawings, in which 
like reference characters designate the same or similar 
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parts throughout the figures thereof. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The accompanying drawings, which are incorporat- 
ed in and constitute a part of the specification, illustrate 
embodiments of the invention and, together with the de- 
scription, serve to explain the principles of the invention. 

Fig. 1 is a block diagram showing the construction 
of an input apparatus according to a first embodi- 
ment of the present invention; 

Figs. 2A and 2B are selection image examples in the 
first embodiment; 

Fig. 3 is a flowchart showing the operation of thefirst 
embodiment; 

Fig. 4 is an example of a selection image for select- 
ing recognition mode(s) according to a second 
embodiment of the present invention: 

Fig. 5 is a flowchart showing the operation of the sec- 
ond embodiment; 

Fig. 6 is a block diagram showing the construction 
of the input apparatus according to a third embodi- 
ment of the present invention; 

Fig. 7 is an example of a similarity table in the third 
embodiment: 

Fig. 8 is a flowchart showing the operation of the 
third embodiment; 

Fig. 9 is a block diagram showing the input appara- 
tus according to the third embodiment; 

Fig. 10 is a block diagram showing the construction 
of a data processing apparatus using the data input 
apparatus of the embodiments; and 

Fig. 11 is a flowchart showing the operation of a 
fourth embodiment of the present invention. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENT(S) 

Preferred embodiments of the present invention will 
be described in detail in accordance with the accompa- 
nying drawings. 

[First Embodiment] 

Fig. 1 shows the construction of an input apparatus 
such as a pen computer, according to a first embodiment 
of the present invention. This construction operates to 



send pen-inputted data to the next process step execut- 
ed by, e.g., an application program 

Fig. 10 shows a data system comprising the data 
input apparatus 1001 which has the construction in Fig. 

£ 1 and a data processing apparatus 1002. In Fig. 10, the 
data processing apparatus 1002 inputs data from an 
data input apparatus 1001. 

In Fig. 1, a locus input unit 1 performs locus-input 
processing to send locus data, inputted from a digitizer 

10 with an input-pen, in recognizable unit, to the subsequent 
processing. A mode discrimination unit 2 sends the input 
locus data to any of recognition units 3 to 5, in accord- 
ance with available recognition mode such as charac- 
ter-recognition mode, figure-recognition mode, corn- 
's mand-recognition mode etc. If a plurality of recognition 
modes are available, the mode discrimination unit 2 
sends the input locus data to plural recognition units. The 
available modes are set prior to locus input by a user of 
the input apparatus, i.e., an application according to 

20 sorts of data which are used by the user. The recognition 
units 3 to 5 respectively confirm the shape of input locus 
and obtain a corresponding code. The figure recognition 
unit 3 determines a predetermined figure similar to re- 
ceived locus data (image), and a figure code indicative 

2S of the kind of figure and similarity data representing the 
similarity between the predetermined figure and the input 
locus data by percentage, to the next processing. The 
character recognition unit 4 sends a character code rec- 
ognized from received locus data and similarity data to 

30 the next processing. Similarly, the gesture recognition 
unit 5 sends a gesture code indicative of the type of com- 
mand and similarity data to the next processing. The rec- 
ognition algorithm and similarity calculation method used 
in the recognition units 3 to 5 may be prior art methods 

35 used in computers as well-known techniques. For exam- 
ple, an input locus is analyzed into line segments, and 
phase-structure features of, e.g., a loop, a dot, concavity 
and convexity are extracted, and the extracted features 
are compared with features of characters or figures. The 

io similarities are given in accordance with the number of 
coincident features. A similarity judgment unit 6 com- 
pares the similarity sent from the plurality of recognition 
units. That is, regarding one image, the similarity judg- 
ment unit 6 determines whether or not the image is a 

45 polychrestic image, from which a unique code cannot be 
recognized in one recognition mode but a plurality of 
codes in respective plural recognition modes can be rec- 
ognized. When the similarity judgment unit determines 
the input image is a polychrestic image, selection image 

so display unit 7 displays the selection images for a user to 
designate a recognition mode. An input data determina- 
tion unit 8 outputs data finally determined as the result 
from recognition or user's selection to the data process- 
ing apparatus 1002 

55 Figs. 2A and 2B show examples of the selection im- 
age displayed by the selection image display unit 7. In 
Fig. 2A, the display has a window 201 displaying a shape 
"X" with a selection button A1 indicating "deletion com- 
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mand", and a window 202 displaying an alphabet "X" 
with a selection button A2 indicating "character". The 
user sees the selection image and knows that the data 
input apparatus 1001 cannot recognize the input, then 
selects the appropriate selection buttons A1 or A2 with 
the input pen. If the user did not intend command input 
and character input and wants to cancel the input itself, 
the user presses an undo button B. Then, the input locus 
data is deleted and the process returns to the state be- 
fore the deleted data was inputted. Also, a recognition 
result can be canceled. As shown in Fig. 2B, the selec- 
tion image has a selection button C indicating "stroke". 
If the user presses this button, only the result from the 
recognition of the input locus data is deleted, and the lo- 
cus data remains in a window 203. For example, in use 
of graphic application, the user may simply input strokes. 
As the recognition processing is not desired, the user 
presses the stroke selection button C to delete only the 
recognition result, and the input data determination unit 
8 outputs the input strokes to the next processing or the 
data processing apparatus 1002. 

Next, the processing in this example where the 
X-shaped locus is inputted with the input-pen and selec- 
tion images in Figs. 2A and 2Bare outputted, i.e., a case 
where character-recognition mode and gesture-recogni- 
tion mode are available will be described with reference 
to the flowchart of Fig. 3. 

Fig. 3 shows the processing that starts when input 
loci are accumulated to a predetermined amount in the 
locus input unit 1 , and performs respective processing 
by the blocks 2 to 7 in Fig. 1 . The locus input unit 1 divides 
the input locusdata with stroke units, asone stroke being 
an interval between a contact time where the input pen 
touches an input surface and a pen-up time where the 
input pen is picked up from the input surface. When a 
predetermined time has elapsed since the stroke input 
started, the locus input unit 1 regards the strokes input- 
ted by that time as one set of symbol-structure data, and 
sends the set of data to the next processing. This lo- 
cus-data dividing may be performed by "character 
cut-out" processing in handwritten character recognition, 
where input strokes are divided into a single character. 
Various well-known techniques are available as charac- 
ter cut-out processing. 

Fig. 9 also shows the construction of the data input 
apparatus 1001 shown in Fig. 10. In Fig. 9, locus data 
hand-written from a digitizer 901 is displayed on a display 
panel 903, and subjected to recognition processing by a 
CPU 902, as a character, a figure or a command. The 
CPU 902 performs necessary processing by executing 
a program stored in a RAM 904 or a ROM 905. The 
processings by the respective units in Fig. 1 are realized 
by the execution of programs by the CPU 902. Note that 
in Fig. 9, in a case where the digitizer 901 comprises a 
transparent member, the digitizer 901 is overlaid on the 
display panel 903 so that the display panel 903 displays 
a locus as if it traces input from the digitizer. This attains 
more natural handwritten input 



In Fig. 3, when a locus is inputted by the locus input 
unit 1 , storage areas rO to r2 for storing similarity data 
are cleared to "0" in step S1 . Next, in step S2, the mode 
discrimination unit 2 checks available recognition mode 
5 (s) in the current application program, and sets a value 
"1 " to a mode flag fg of the available recognition mode. 
The flag values arefg(O), fg(1 ) and fg(2), respectively for 
figure recognition, character recognition and gesture 
recognition. In this embodiment, these flags are pre- 

10 pared in accordance with recognition processings by the 
application program To perform recognition of pen input 
in use of application program or utility program, recogni- 
tion processings can be determined in designing of the 
program, and set in advance as this embodiment. For 

'5 example, when an application which functions as a note 
pad is triggered, the application accepts only character 
input and gesture input, accordingly, the application sets 
the necessary recognition modes to a memory area. The 
flags fg(0) to fg(2) are set corresponding to the stored 

20 contents. In a case where a selection image is displayed, 
as shown in Figs. 2A and 2B, as character recognition 
and gesture recognition are available, the flags fg(1 ) and 
fg(2)aresetto"1". 

The process proceeds to step S3 to SS, in which the 

2S respective flags fg(0) to fg(2) are checked and recogni- 
tion processing corresponding to a set flag value is exe- 
cuted to the set of strokes sent from the locus input unit 
1 . As a result, a recognized code and similarity between 
the recognized code and the input data, i.e., the reliability 

30 of the recognition result are obtained. As mentioned 
above, the similarity data is represented by percentage 
where "100" means complete match, and "0" means the 
opposite. In this example, in steps S5 and S7, it is deter- 
mined that the values of flags fg(1 ) and fg(2) are "1 ", and 

35 character recognition and gesture recognition are re- 
spectively performed in steps S6 and S8. Then, values 
indicating the recognition results are respectively set as 
a recognized character codel , its similarity r1 , a recog- 
nized gesture code code2, and its similarity r2. In this 

40 example, an alphabet letter "X", is set as the code 1 , a 
deletion gesture code is set as the code 2, and "85" is 
set to the similarity r1 , and "70" is set as the similarity r2. 
As the value of the flag fg(0) is "0", figure recognition is 
not performed, and a recognized figure code codeO and 

43 its similarity rO have no value. Note that "0" of the simi- 
larity r0 means " no match". 

Next, in step S9, the maximum value among the sim- 
ilarities rO to r2 are stored as mr. In this example, as the 
similarity r1 (85) is the greatest value, the value 85 is 

so stored as mr. In step S10, whether or not the mr value is 
greater than a predetermined threshold value is deter- 
mined. In this example, value "70" is set as a threshold 
value between relatively high values and relatively 
not -so high values. If the maximum similarity mr is equal 

ss to the threshold value or greater, i.e., recognized result 
has a predetermined or higher reliability, the process pro- 
ceeds to step S11 , while if not, it is determined that no 
code has been recognized, and the process ends Note 
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that if the similarity is equal to the predetermined value 
or greater, it is considered that the recognition has been 
successful. 

In step S11 , whether or not there is any similarity r 
closest to the maximum similarity mr is checked. In this 
example, the gesture similarity r2 is determined as the 
closest similarity. This determination is made based on 
existence/absence of another similarity within a prede- 
termined range. In this example, the predetermined 
range is "20". In this manner, if it is determined that there 
is another recognized code in another recognition mode, 
the process proceeds to step S12 in which a selection 
image is outputted, then, as the user selects one mode, 
the recognition result in the selected mode is stored in a 
recognized-code storage area rcode, and the process 
ends. If there is no other similarity in step S11, the proc- 
ess proceeds to step SI 3 and the code of the maximum 
similarity mr is stored into the recognized-code storage 
area rcode, and the process ends. 

Next, the case in Fig. 2B will be described in accord- 
ance with step S3 and the subsequent steps. 

In this example, character recognition and gesture 
recognition are performed (fg(1 ) - 1 andfg(2) - 1). Then, 
character recognition is performed and a character "X" 
is recognized. A character code "X" is stored as the rec- 
ognized character code codel, and a similarity value "85" 
is set as the similarity rl (step S6). 

Also, gesture recognition is performed and a dele- 
tion command is recognized A deletion command is 
stored as the recognized gesture code code2, and a sim- 
ilarity value "70" is set as the similarity r2 (step S8). 

Next, the similarity value r1 (85) is set as the maxi- 
mum similarity mr (step S9), and whether or not the max- 
imum similarity mr value is 70 or greater (step S10). As 
mr = 85 > 70, whether or not there is recognized result 
with a similarity of 65 ((mr-20) = (85-20)) or greater is 
examined (step S11). Since the similarity value r2of the 
gesture recognition result is 70 and satisfies the above 
condition, the selection image in Fig. 2B is displayed 
(step SI 2). The user selects the "character" selection 
button A2 if he/she user wants to input "X" as a character, 
while selects the "deletion command" selection button 
A1 if he/she wants to input "X" as a command. If the user 
wants to input "X" as strokes, he/she selects the "stroke" 
selection button C. Then, a selected code is stored in the 
area rcode as input data. Note that if the "stroke" button 
C is pressed, no code is inputted. 

Thus, if the system cannot make clear determination 
among discriminate command recognition (gesture rec- 
ognition), character recognition, figure recognition, or 
stroke input, a selection image is displayed so that the 
user can select recognition mode. Accordingly, in each 
recognition mode, it is not necessary to consider any 
possible recognition code in another recognition mode. 

Note that the similarity in the present embodiment 
may be replaced with any value so far as it indicates re- 
liability of recognition. Further, the range of the value may 
be arbitrarily set. Since the present embodiment needs 



only rough reliability judgment, the values used in steps 
S10 and S11 may be set corresponding to the level of 
reliability. 

When a plurality of recognition processings other 
s than the above processings are used, the similarities r, 
the recognized codes rcode, mode flags fg may be in- 
creased in accordance with recognition processings. 

[Second Embodiment] 

Fig. 4 shows a selection image for setting the rela- 
tion among a plurality of recognition processings, of the 
data input apparatus 1001 according to a second em- 
bodiment of the present invention. This embodiment ba- 

15 sically has the same construction as that of the first em- 
bodiment, except that the operation in the flowchart of 
Fig. 5 is different in some steps from that of Fig. 3. The 
difference from the first embodiment will be described 
with reference to Figs. 4 and 5. 

20 in Fig. 4, a slide 10 is used for setting parameter of 

displaying a selection image. The parameter is a refer- 
ence value for determining the similarity. When the slider 
1 01 is moved leftward, a selection image is not displayed 
often even if a plurality of codes with in a plurality of rec- 

2S ognition modes arc picked upas very similar to each oth- 
er. On the other hand, when the slider 1 01 is moved right- 
ward, a selection image is often displayed even if 
picked-up codes are not so similar to each other. Thus, 
the range value (20 in step S1 1 ) for comparing the max- 

30 imum similarity mr with its close value is changed by ma- 
nipulating the parameter setting slide 10. In the first em- 
bodiment, this range value is fixed, however, in this em- 
bodiment, a value set by the slide 10 is stored intoastor- 
age area, and it is read out of the storage area at each 

35 processing. For example, assuming that the value of the 
left end position of the slide 10 is "0" and that of the right 
end position is "40", a value between these set values is 
linearly set depending upon the position of the slider 1 01 
and stored in a storage area rr. InstepSH' in Fig. 5, the 

40 value is read out of the storage area rr, and whether or 
not a value (mr-rr) exists is examined. In this manner, the 
user can arbitrarily change the parameter for selec- 
tion-image displaying. 

In Fig. 4, numerals 11 to 1 3 denote areas for setting 

45 priority of respective recognition modes. The priorities 
are determined by respectively inputting numerals cor- 
responding to the areas. For example, if the areas 11 to 
13 all have a value "100", gesture recognition, character 
recognition and figure recognition all have the same pri- 
se ority When the areas 11 to 13 have values "100", "70" 
and "70" respectively, gesture recognition has a higher 
priority than other recognition modes. The input values 
are percentage values to be used in multiplication of the 
recognition similarities. Actually, in step S8.5, the re- 

55 speclive recognition similarities are multiplied by the set 
percentages. In a case where the percentages are 100: 
70: 70, the recognition similarities r are, rO= rO * 0.7, r1 
= M * 0.7, r2 = r2 * 1.0; i.e., the similarities of figure rec- 
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ognition and character recognition are lowered, and as 
a result, an input locus is likely to be recognized as a 
gesture command. Note that if there are codes very' sim- 
ilar to each other, a selection image is outputted for the 
user to select a recognition mode, similar to the process- 
ing in Fig. 3. In Fig. 4, input of percentage values are 
made before locus input. 

Fig. 5 shows the operation as described above in 
detail. The processing in Fig. 5 is basically the same as 
that in Fig. 3 except additional steps S8.5, S10.5 and 
modified step S11'. In step S8.5, to control the priorities 
of recognitions, the respective similarities are multiplied 
with input percentage values. In step S10.5, to control 
selection-image display frequency, a set value is read 
out of the storage area rr. In step SIT, a similarity range 
mr-rr is used. 

This processing allows the user to select one of com- 
mand recognition (gesture recognition), character recog- 
nition, figure recognition and stroke input, if the system 
cannc» clearly discriminate available recognition mode 
(s) wit; : respect to an input locus. Accordingly, in each 
recognition processing, it is not necessary to consider 
other possible recognized shapes in other recognition 
process. In addition, in this embodiment, the user can 
arbitrarily set the selection-image display parameter and 
the priorities of a plurality of recognition modes. 

[Third Embodiment] 

Fig. 6 shows the construction of the data input ap- 
paratus 1001 according to a third embodiment of the 
present invention. 

In Fig. 6, the respective blocks look similar to those 
in Fig. 1 , however, the construction is different from that 
of Fig. 1 . This embodiment performs gesture recognition 
for all locus inputs, further avoids confusion of gesture 
recognition 

In Fig. 6, a locus input unit 21 corresponds to the 
locus input 1 in Fig. 1; a mode discrimination unit 22, to 
the mode discrimination unit 2; a gesture recognition unit 
23, to the gesture recognition unit 5, a selection image 
display unit 25, to the selection image display unit 7; a 
figure recognition unit 26, to the figure recognition unit 3; 
and a character recognition unit 27, to the character rec- 
ognition unit 4. Note that the similarity judgment unit 24 
is different from the similarity judgment unit 6 in opera- 
tion. 

Fig. 7 shows a similarity table used by the similarity 
judgment unit 24. The similarity table is included in the 
similarity judgment unit 24 In Fig. 7, a column 71 con- 
tains input loci; column 72, gesture codes; column 73, 
figure codes for discriminating figure type; and column 
74, character codes. For simplicity of explanation, Fig. 7 
conceptually illustrates the table with shapes instead of 
actual codes. Note that the input-locus column 71 does 
not exist in an actual similarity table. Fig. 7 shows the 
column 71 merely to show the corresponding codes for 
each input. 



Fig. 8 shows the processing procedure according to 
the third embodiment, i.e., processings by the gesture 
recognition unit 23 to the character recognition unit 27, 
to obtain a recognition result. The processing in Fig. 8 is 
5 realized by executing a program by the CPU 902 in Fig. 
9. The operation of the third embodiment will be de- 
scribed below with reference to the flowchart of Fig. 8. 

In step S800, the mode flags fg(0) to fg(2) are set in 
a similar manner to that in step S2 of the first embodi- 
io ment. 

In step S801, the gesture recognition unit 23 is ac- 
tivated, and a gesture code code2 and its similarity r2 
are set as the recognition results. Next, the process pro- 
ceeds to steps S802 and S803 for similarity judgment. 
15 In step S802, the similarity is checked. If the similarity r2 
which is the reliability of the gesture recognition result is 
over a set threshold "70", i.e., the reliability is high, the 
process proceeds to step S803 in which the similarity ta- 
ble as shown in Fig. 7 is searched. This search is made 
20 by examining whether or not a gesture code in column 
72 coincides with the code2. For example, when a locus 
701 is inputted and a code 702 is recognized as the rec- 
ognition result, it is determined that a figure code 703 
and a character code 704 correspond to the input locus 
25 701 . Then, these codes 703 and 704 are set as codeO 
and codel, and the process proceeds to step S804, in 
which the result of the table search, i.e., whether similar 
codes are retrieved or not is determined. If YES, the 
process proceeds to step S805 in which a selection im- 
30 age is outputted, so that the user selects a desired rec- 
ognition mode. The code of the selected mode is deter- 
mined as the final recognized code, and the process 
ends. The selection image may be as shown in Figs. 2A 
and 2B. 

35 If it is determined in step S802 that the recognition 

reliability is not high, the process proceeds to steps S806 
and S808, in which the mode flags fg(0) and fg(1) are 
checked, and figure recognition in step S807 or charac- 
ter recognition in step S808 is activated. Then, the rec- 
40 ognized code is determined as the final recognition re- 
sult, and the process ends. 

Thus, one shape can be commonly used in a plural- 
ity of recognition processings such as character recog- 
nition, command recognition and gesture recognition. Al- 
45 though the present embodiment uses only one recogni- 
tion processing, i.e., the gesture recognition, the present 
embodiment enables higher processing than the 
processing in the First embodiment, since the first em- 
bodiment activates all the recognition modes, while this 
so embodiment perform recognition processing on one 
symbol-structure data by one recognition mode. Further, 
the combination of the third embodiment with the first 
embodiment or the second embodiment will attain con- 
current use of plurality of recognition processing with 
55 high speed. 
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[Fourth Embodiment] 

Next, the data input apparatus 1001 according to a 
fourth embodiment, in which a plurality of recognition 
modes can be used, will be described. The input appa- 
ratus has the same construction as that of the third em- 
bodiment shown in Fig. 6, and has a similarity table as 
shown in Fig. 7. The difference from the third embodi- 
ment is the processing procedure shown in Fig. 11 . The 
processing procedure according to the fourth embodi- 
ment will be described with reference to Fig. 11. 

In Fig. 11, the process starts when a set of symbol 
data is inputted. In step S1100, the mode flags fg(0) to 
fg(2) are set in a similar manner to that in step S2 of the 
first embodiment. In step S1101, gesture recognition is 
performed, and the obtained code and its similarity are 
set as the recognized gesture code2 and the similarity 
r2. In step S1 1 02, the similarity value is examined wheth- 
er it equal to a set threshold "70" or greater. If it is "70" 
or greater, the similarity table is searched in step S1103. 
The retrieved code is examined in step S1 104, and if an- 
other code with high similarity exists, these codes are 
displayed in the form as shown in Fig. 2A or 2B for the 
user's selection. Then, the determined code is set as the 
final recognitfon result rcode. If there is no other code, 
the gesture code code2 is determined as the final recog- 
nition result. 

On the other hand, if the similarity r2 is less than the 
threshold "70", whether or not figure recognition mode is 
set is examined in step S1106 If YES, figure recognition 
is performed in step S11 07, and the obtained code and 
its similarity are set as the recognized figure code codeO 
and the similarity rO. Next, the similarity rO is examined 
in step S1 1 08, and if it is "70" or greater, the figure codeO 
is determined as the final recognition result. 

If gesture recognition and figure recognition have 
not obtained a final recognition result, whether or not 
character recognition is set is examined in step S1109. 
If YES, character recognition is performed in step S1 110, 
and the obtained code and its similarity are set as the 
recognized character code codel and the similarity r1. 
If the similarity r1 is "70" or greater, the code 1 is deter- 
mined as the final recognition result. 

If the similarity r1 is less than "70", it is determined 
that there is no code corresponding to the input sym- 
bol-structure data, and the process ends. 

In this processing, if a gesture code is recognized, 
the table is searched for code of similar shape, thus at- 
taining high-speed processing If a recognition result 
having a similarity "70" or greater is not found, the 
processing moves from the current mode to another rec- 
ognition mode, thus performs gesture recognition, figure 
recognition and character recognition, with respect to 
one input. 

The present invention can be applied to a system 
constituted by a plurality of devices, or to an apparatus 
comprising a single device Furthermore, the invention 
is applicable also to a case where the object of the in- 



vention is attained by supplying a program to a system 
or apparatus 

The present invention is not limited to the above em- 
bodiments and various changes and modifications can 
5 be made within the spirit and scope of the present inven- 
tion. Therefore, to appraise the public of the scope of the 
present invention, the following claims are made. 



10 Claims 

1. An image recognition apparatus for recognizing an 
input locus, comprising: 

locus input means (1, 21) for inputting locus; 
15 recognition means (3, 4, 5, 23, 26, 27) for rec- 

ognizing an input image constituted with the locus 
inputted by said locus input means; 

judgment means (6, 24) forjudging whether or 
not the input image can be recognized as a plurality 
20 of functions; and 

selection means (7, 8, 25, 28) for, if said judg- 
ment means judges that the input image can be rec- 
ognized as a plurality of functions, selecting one of 
the plurality of functions as a recognition result. 

2. The image recognition apparatus according to Claim 

1 , wherein said recognition means has a plurality of 
recognition modes corresponding to types of input 
image and performs recognition by each mode, and 

30 wherein said judgment means judges similarity 
between a candidate shape and the input image in 
each recognition mode. 

3. The image recognition apparatus according to Claim 
35 2, wherein the plurality of recognition modes include, 

command recognition mode (5, 23) for recognizing 
an input image as a command, figure recognition 
mode (3, 26) for recognizing an input image as a fig- 
ure and character recognition mode (4, 27) for rec- 
40 ognizing an input image as a character. 

4. The image recognition apparatus according to Claim 
3, further comprising memory means (904, 905) for 
storing correspondence (72, 73, 74) among the plu- 

4£ rality of functions, wherein said recognition means 
first performs recognition in the command recogni- 
tion mode, and if the recognition is successful, said 
selection means refers to the correspondence in 
said memory means to select one shape 

so 

5. The image recognition apparatus according to Claim 

2, further comprising priority input means (11, 12, 
1 3) for inputting priorities of the respective recogni- 
tion modes, wherein said judgment means performs 

55 judgment of functions by using the priorities. 

6. The image recognition apparatus according to Claim 
2, wherein said recognition means adds a predeter- 
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mined order to the respective recognition modes, 
and if recognition in one mode is unsuccessful, per- 
forms recognition in another recognition mode, in 
accordance with the order. 

7. The image recognition apparatus according to Claim 
6, further comprising memory means (904, 905) for 
storing correspondence (72, 73, 74) among the plu- 
rality of functions recognized in the respective rec- 
ognition modes, wherein said recognition means 
first performs the recognition in the input image in 
the command recognition mode, and if the recogni- 
tion is successful, said selection means refers to the 
correspondence in said memory means to select 
one of the similar shapes. 

8. The image recognition apparatus according to Claim 
6, wherein if the recognition of input image is suc- 
cessful in one recognition mode, said selection 
means selects a recognized shape from the recog- 
nition as a recognition result. 

9. The image recognition apparatus according to Claim 
2, further comprising display means (903) for dis- 
playing an image, wherein said selection means dis- 
plays the plurality of shapes recognized in the 
respective modes, judged by said judgment means 
as functions for selection of one shape. 

10. The image recognition apparatus according to Claim 
9, further comprising reference input means (101) 
for inputting a reference value of similarity, wherein 
selection means displays the functions in accord- 
ance with the reference value. 

1 1 . The image recognition apparatus according to Claim 

I, wherein said recognition means obtains similari- 
ties between the input image and shapes corre- 
sponding to the recognized functions, and wherein 
said selection means selects one of the functions 
based on the similarities. 

12. The image recognition apparatus according to Claim 

II, wherein if the difference between similarities 
between the plurality of shapes judged by said judg- 
ment means as functions and the input image is less 
than a predetermined value, said selection means 
displays the plurality of shapes for selection o( one 
shape. 

1 3. The image recognition apparatus according to Claim 
12, further comprising input means (101) for input- 
ting the predetermined value. 

14. The image recognition apparatus accordingtoClaim 
11, further comprising priority input means (11, 12, 
13) for inputting priorities, wherein said judgment 
means performs weighting on the similarities in 



accordance with the priorities. 

15. An image recognition apparatus comprising: 

locus input means (1. 21) for inputting locus; 

5 recognition means (3, 4, 5, 23, 26, 27) for rec- 

ognizing an input image constituted with the locus 
inputted by said locus input means in a plurality of 
recognition modes in accordance with meaning of 
the input image; 

io judgment means (6, 24) for judging similarities 

between corresponding functions recognized in the 
plurality of recognition modes and the input image; 
and 

selection means (7, 8, 25, 28) for displaying 
15 the shapes judged by said judgment means as func- 
tions for selecting designated one from the shapes 
as a recognition result; 

16. An image recognition apparatus for recognizing an 
20 input locus, comprising: 

locus input means (1, 21) for inputting locus; 
recognition means (3, 4, 5, 23, 26, 27) recog- 
nizing an input image constituted with the locus 
inputted by said locus input means in a plurality of 

25 recognition modes in accordance with type of the 
input image; 

memory means (904, 905) for storing relation 
among similarities of shapes recognized in the plu- 
rality of recognition modes; 

30 retrieval means (6. 24) for, in a case where 

said recognition means has recognized the input 
image as a shape in a predetermined one of the plu- 
rality of recognition modes, retrieving a shape anal- 
ogous to the shape recognized in the predetermined 

35 recognition mode from the shapes recognized in the 
other recognition modes; and 

selection means (7, 8, 25, 28) for displaying 
the shapes retrieved by said retrieval means for 
selecting designated one of the shapes as a recog- 

40 nition result. 

17. An image processing method for recognizing an 
input locus, comprising: 

a recognition step (S4, S6, S7) of recognizing 
■*5 an input image as shapes having different functions, 
in a plurality of recognition modes; 

a judgment step (S11) of judging similarities 
between the shapes recognized in the plurality of 
recognition modes in said recognition step and the 
so input image; and 

a determination step (S12) of, if it is deter- 
mined in said judgment step that there are functions, 
displaying the functions, and determining selected 
one of the shapes as a recognition result. 

18. An image processing method according to Claim 1 7, 
wherein in said recognition step, the plurality of rec- 
ognition modes are command recognition mode for 
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recognizing an input image as a command, figure 
recognition mode for recognizing an input image as 
a figure and character recognition mode for recog- 
nizing an input image as a character. 



by the image recognition method in Claim 17; and 

input means for inputting a code correspond- 
ing to a shape obtained as a recognition result from 
recognition by said recognition means. 



19. The image processing method according to Claim 
17, wherein said recognition step includes a step 
(S10) of judging whether or not recognition is suc- 
cessful. 

20. The image processing method according to Claim 
17, wherein in said recognition step, a similarity 
between an input image and candidate shapes are 
obtained, and in said judgment step, analogies 
among the recognized shapes are judged by com- 
paring similarities between the recognized shapes 
and the input image respectively. 

21. The image processing method according to Claim 

17, further comprising a step ol inputting priorities, 20 
wherein in said judgment step, the similarities are 
judged using the input priorities. 

22. The image processing method for recognizing an 
input locus, comprising: 2S 

a recognition step (S801 , S1101) of recogniz- 
ing an input image as a first type shape; 

a retrieval step (S803, S1103) of retrieving a 
shape, similar to the first type shape recognized in 
said recognition step, from a table holding in 30 
advance relation of analogies among shapes; and 

a determination step (S805, S1105) of, if a 
shape similar to the first type shape is retrieved in 
said retrieval step, displaying the first type shape 
and the shape retrieved in said retrieval step, and 3s 
determining the selected one of the shapes as a rec- 
ognition result. 

23. The image processing method according to Claim 

22, further comprising a second recognition step 40 
(S807, S1107) of, if the recognition in said recogni- 
tion step is unsuccessful, recognizing the input 
image as a second type of shape. 

24. The image processing method according to Claim *s 

23, furthercomprisingathird recognition step(S808, 
Si 110) of, if recognition in step second recognition 
step is unsuccessful, recognizing the input image as 
a third type of shape 

so 

25. The image processing method according to Claim 

24, wherein in the table, shapes of the first type, the 
second type and the third type are in correspond- 
ence as functions. 



27. A data input apparatus for encoding an input locus 
and inputting code data, comprising: 

recognition means for recognizing an image 
by the image recognition method in Claim 22; and 

input means for inputting a code correspond- 
ing to a shape obtained as a recognition result from 
recognition by said recognition means. 

28. Data input apparatus including locus-sensitive 
means for entering data, and means for deciding the 
probable nature of the data. 



26. A data input apparatus for encoding an input locus 
and inputting code data, comprising: 

recognition means for recognizing an image 
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